A Derivative-Free Algorithm for Least-Squares Minimization

A Derivative-Free Algorithm for Least-Squares Minimization

We develop a framework for a class of derivative-free algorithms for the least-squares minimization problem. These algorithms are designed to take advantage of the problem structure by building polynomial interpolation models for each function in the least-squares minimization. Under suitable conditions, global convergence of the algorithm is established within a trust region framework. Promisi...

On the local convergence of a derivative-free algorithm for least-squares minimization

In Zhang et al. (accepted by SIAM J. Optim., 2010), we developed a class of derivative-free algorithms, called DFLS, for least-squares minimization. Global convergence of the algorithm as well as its excellent numerical performance within a limited computational budget was established and discussed in the same paper. Here we would like to establish the local quadratic convergence of the algorit...

A Derivative-free Algorithm for Finding Least Squares Solutions of Quasi-linear and Linear Systems

A novel derivative-free algorithm for solving quasilinear systems is presented. It resembles “classical” optimization approach but greatly simplifies computation, resulting in fast execution and numerical stability. Though the global convergence cannot be guaranteed, it turns out that the presented algorithm finds a solution as successfully as other commonly accepted methods. The algorithm is c...

Solving Derivative-Free Nonlinear Least Squares Problems with POUNDERS

Least-Squares Minimization Under Constraints

These algorithms are, however, not designed to perform least-squares minimization under hard constraints. This short report outlines two very simple approaches to doing this to solve problems such as the one depicted by Fig. 1. The first relies on standard Lagrange multipliers [Boyd and Vandenberghe, 2004]. The second is inspired by inverse kinematics techniques [Baerlocher and Boulic, 2004] an...